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ABSIBACI 

Thirty^three novices and 33 expert calculator U3^rs 
took a test in which they predicted vhat number vould be in the 
calculator's display after a sequence of button presses (such as 2 
3 Subjects* answers did not depend on the brand of calculator 
they actually owned. Simple production systems models were fit to the 
performance of each subject. Subjects differed greatly with respect • 
to when they thought an expression vould be evaluated, the order in 
which a chain of calculations was evaluated, and whether the display 
would be incremented. (Author) 
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3Bsf r act 

lUtcty-three novices and 33 ^LmuLxl aic^czd^tor jsssrs took 
:3sreri±cted what number would ^ -ra^mlytcair^ dispSt^z 
3>nr^rm pEZssses (such as 2 f)* :Si^9Xs* aamers dii niBt (Wemf arr^tfe^ 
^)BM± o£ ryrlnnlator they acti^iLvwBEE. SiofjC^ixacodiiccloa^'^rstems iiiiiii 1ii ^^gere 
f Lli u> tie^p^Bformance of eacdhr -fiaaj^ect:^. Sobf^iCtts ftfi.f f easA g^Rjrf^ xaafcttzr n>WM < t 
Cfir Allien idi^tS^ught an expi^SBKf -^ '^wa^Jx^ lardfessared^ scde 
f^hfeSB of GflabOEations was eii iiTIii<ffffe^i. im flu i rtie-o^^plsT^-scnLd 
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Analysis of Stndents' ItrtuirioDESL About: the X^e r a ri on -ost gfacrtronic ^ - al ml arn rs 
RICHARD £• 1S5ER and PISAYE BAYHAN, Hofversfcy of CalfiJaEHia, Santa Barbara 

Objectives 

Durii^rtfe past decaftr> -there Ihk beati-axde^Ecead safi rapid accseptance of 
electronic osfealators in scxriety* Alaroogji calcu3«nns are b^^ on a tech- 
nology thaaz Sm not reach aflJ^siHarlaets uwr^ late in 1971 fMullish, JL976), elec- 
txonic cala!a^:x>rs are hee^^^amg a ciMMm feature of ourascion's classrooms. 
Based on a ^anrey of artixStes^and edicprial* poblfshpd TgfTlrrn the past f esy years 
in Arithm^^gfe Teacher axs iSSs^jF"'^'^^^ ^3eacter^ as ^vell assrpolicy s^satemnnt by 
the NationaEL Council of 3«&cbe32S: of V il liiiiinl f i (1376), ±cras clear Ciiat calcu- 
lators wilLpiay an imp 
example, in recent papec: 
press has any invention 
ticularly matiiematics e 

However, in spite of 
ments, the research co) 




xn de^ adncacaait of 
Kryfitfc ;1976) sLa4=p 
iiiJi.mriPl Sspc: reaPE LiiLiwuit?. 1 1 > g edi 



can stuESo^s^ 3?br 
''Not STTrrp tfae i 



eothu 



"endorse— 



very sLov £2 pmcSing: inroi:iiiHt Jtorr t-lm^r 
-"^E^ useful in inspenatixg -raffi iHLjiifrir'^urrfmlqw^ revofiiitiaBi. The jzaasssive 
deqelopmerrts in improveddhardnare teHe^ tiot keen notched C33mparat±0e^ ^eveljop- 
ments in what Shneiderman (19913 3oad3s ^isoCcware ps^cfaoSogy^'' This sa^, 
we know very little about hoi 1 injlfii ^sme qd uafcLbLa nd aloxlators, mnp^ t^^pes 
of instruction will help peopfedbMtf.i>M lurratlve vKxs, ^«br ^Knoe pec^tt sBBsoLto 
never use them well, ooc how to^dosl^^operacing ^jpSilwits t&tf^^make ji ipiliBTiinji,! i. i1 
sense. Since the calaxLaZOT-^seammamBs a s t ud e ntt^s rrrs t: ^rm efa n f hw i a. 
computer, to a computes: lang^c^i^ sanfi ts> ccspocer nrviLy :3S£:gegHr7rT.-,^±t: ±s 
important to understand how insnoB oise and mdescstand caSKame£:oTSi^ 

The ob j ective of Tfre presenc saiti^ wee so detpmrinff tibei knowfrdgr liw; 
ozxlinary. users (novices) and .soffediKlcatjQfl nsexs (e3cpertBF) love copcegggig : jJal 
operation of hand held jcalculaxas:^ 2b parirrcttlar tfrp .gftftt was to descriise: each 
s^ject^s conception uIT-U ie c£bcs±a£Ar's {^eeatrin;^ Ad then tu iiih^tp crnrn- 

parisons between experts and wixass. 



Perspectives 

In order to proracte a f ooanal 3^»crlpt:><m of each issS^sf knowcLedge, we inre 
cteareloped a productioitiisyptenr (Neaaell & Smm^ 1972) feir«ch user^ The cot- 
dxtion for each production is:sa kear pcsess saeSi as pr ps«rn w g; a plus k e y (+) afrer 
pressing a digit key (e.g., 37. Ikfc^gtiflB £or csech i i nMh ii r ion^ a change 
.jr no change) in the status tof T 1^ ylfM BB'iljy aari/ar t iier pwifPm iaL registers the 
c^culator. There are altemativae^actittas «Mclfe may he associ ated with eac fr 
condition; for example, if the stodent aeha ntc s » if ^ plus acts like an ecgnals 
then the answer to 3 + 2 + is 5, but dtf MiNfcoiL beiiaves as if a plus does not 
act like an equals then the answer to 3 ^ 2 -3- ^ 2. In the former case, tiie 
display is set to the evaluated value of t&e cz^cessinn in the register (D = eval 
(R)) but in the latter there is no daa^ Jm. tim. display for pressing aTtw 
2 (D - D) . The present study focused on ^S^^^f^&rentrrconditionsr number after 
+, + after number, + after +, + after ^sr -mn^. mtmh^ ^ = after -H, ntmiber after 
x, X after number, = after x, x after x^rrMhM x, xi^after +, -i-^after x. Eor 
each condition, alternative actions woe es^Hped, sorthat ea ch su bject's krrow- 
ledge could be represented as a list -of X3^'3QEatMC!tions>«> 



Data Source 

Thirty-three novice users were 
the University of California, Santa Bsrbaca;^ 



tte Psychology Subject Pool at 
;e sabjects had no experience 



ERIC 



«CLth csmpm&rs or computer programming, and were selected out of a larger sample 
of 463X0&zises because they gave consistent performance in the experiment. Thirty- 
ticree &^BXs were recruited from Computer Science majors at the University of 
GalifomiJi^ Santa Barbara. These subjects were taking an intermediate level 
5n:ogramm±^ course that included analyses of operating systems, and were selected 
TiTTt of aijfeETger sample of 35 experts because they gave consistent performances in 
xiie expezi^tent. 

Method 

A sample of 33 novice users and a sample of 33 expert users were given a four 
page typecritten questionnaire. The questionnaire contained 88 items such as: 
2+3, 2 + 3-:, 2 + =, 2 + 3 + «, 2 + 3 x, 2++, 2 + = +, 2+ = + 3, 
2 + =+ = + =, 2x+5=, 2x = x= x=. For each item the subject was asked to 
write down the number that would be in the calculator's display after the last 
key was pressed (assuming the calculator was cleared at the start of the problem). 
The introduction made clear that the user should assume that the calculator is a 
typical simple one (not a EBP calculator that uses reverse Polish notation). 

Rfesults and Conclusion 

The data for each subj«:t consisted of 88 numbers, i.e., the answers given. 

(1) Relip.bility. Therquestionnaire consisted of two 44-item forms of the 
same calculator operations (such as 2 + 3 + = on one form corresponding to 
7 -r 3 + = on the other). The number of times a subject failed to give corres- 
ponding answers cn the 44 pairs of problems was tallied (e.g., corresponding 
answers for the above two problems could be 3 and 3 respectively, or 5 and 10 
respartively, or 10 and 20 respectively, etc.). The experts were far more con- 
sistent than the novices. However, the samples lised for further analysis consist 
only oof subjects who are highly consistent in their performance (n = 33 for 
each ^oup) ^ 

£2) Comparison with user's calculator. Most of the subjects in our sample 
owned calculators so the answers given by each of the major models on our test 
was compared to the performance of each subject. For both experts and novices, 
Texas liistruments (TI) models produced answers which corresponded to more of our 
subjects' answers than any other models. This was true for subjects who owned 
TI calculators and equally true for subjects who owned other models or who owned 
none. It was also equally true for subjects who used their calculators often 
as coii5)ared to those who used them infrequently. Thus, the present results suggest 
that the operating system in simple TI models best fits the intuitions of human 
users. However, the answers given by th^ TI models failed to match the answers 
given by human subjects on an average of 20% of the problems for experts and 
19.5% for novices. Thus, it is not sufficient to conclude that people "think 
like Tl's operating system." 

(3) Fitting a production system to the performance of each subjects- In 
order to provide formal descriptions of each user's conception of the operating 
system of a calculator, individual rproduction systems were fit to the performance 
of each user. First, only the data for problems involving 6 conditions (with 
number, plus, and equals) was analyzed. Then, three more productions were added 
to each subject based on performance on problems with multiply and equals. Fin- 
ally, 4 more productions were added based on the problems with conditions in- 
volving nnu s"^'^ sequences of plus and multiply. 

As an example of how the analysis was conducted, the following table gives 
the typical patterns of performance on problems involving only 6 simple con- 
ditions (problems involve only n?rnber, plus, and/or equals). 
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Question Group 1 
Answers 



2 


2 


Zt 


2 




3 


zrrjrr 


c 




7 


Z= 


2 


2+= 


2 


2+3= 


c 


2+3+= 


5 


2+3+7= 


12 


Pit 
2++ 


2 


2+^ 


2 


2I°I°I 


2 


2^3 


3 


2+4« 


2 


24«+= 


2 


2-I"l=°l = 


2 


2+-H-3= 


5 



Grocp 2 


Group 3 


Anseers 


Answers 


2 


2 


2 


2 


3 


5 




5 


7 


12 


X 


2 


z 


2 


5 


5 


5 


5 


12 


12 


2 


2 


2 


- 2 


2. 


2 


3 


5 


2 


2 


2 


2 


2 


2 


5 


5 



Group 4 


Group 5 


Answers 


Answers 


2 


2 


2 


2 


3 


3 




c 


7 


7 


2 


2 


4 


2 




5 


Iv 




12 


12 


4 


4 


4 


2 


8 


2 


3 


3 


8 


4 


8 


2 


16 


2 


7 


5 



Groop 1 corresponds to the answers given by TI models; expressions are evaluated 
when an = or a + key is pressed^ Group 2 is identical except that expressions are 
evaluated only when an =* key is i)ressed. Group 3 is identical except that ex- 
pressions are evaluated when a nnmber, a + or an = is pressed. Group 4 corresponds 
more closely to a Rockwell calcoiator and increments the display in cases where 
= follows +, or when + follows +. Group 5 is like Group 4 except that the display 
is incremented only when two pluses occur. There were 8 subjects in Group 1, 
10 in Group 2, 5 in Group 3, 2 in Group 4, 1 in Group 5, and 7 miscellaneous 
patterns for novices; corresponding numbers for experts were 11, 7, 2, 4, 3, and 
5 respectively. For example, for Group 1, + after number results in evaluating 
and displaying the expression but not in Group 2; or + after + results in in- 
crementing the display for Groups 4 and 5 but lOt for tiie other groups. More de- 
tailed analysis will be presented in the full paper; major differences involve 
when an expression is evaluated, the order in which a chain of calculations was 
evaluated, and whether the display would be incremented for + being pressed 
after +, x after x, = after +, and = after x. 

Educational and Scientific Sigrrf -H cance 

The present study is an attenpt to apply the formsJL analysis tools oe cog- 
nitive psychology to the real-woBd.d problems of classroom education. This work - 
provides new information concerning how people understand and think abou^ 
electronic calczilators, and thus^ontributes to a growing theory of compn£sr 
literacy. In addition, this woric has implications for instruction since 2t: is 
clear that self-taught users u^ffer greatly in their intuitions (and hence 
in their effectiveness ais users); this work also has practical implicatioBs 
for design of operating systems ^lat are consistent with human intuitions^ 
Finally,- it is hoped that this work will serve as a stimulus for future research 
in helping sttidents to get the mo^ out of an exciting new technology. Saice cal- 
culators are r. student's first introduction to computers, it is important ^at 
students see that the operation of the calculator can be imderstood. 
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